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ABSTRACT 

This study surveyed 199A graduates (N=91) of the 
University of Kentucky College of Medicine (UKCM) regarding their 
satisfaction with the medical education they received at the school. 
The Class of 199A was the last to finish before the implementation of 
a c- '■'■iculum reform initiative designed to increase active student 
learr.ing, integrate classroom and clinical knowledge, and promote 
self-assessment and life-long learning. Graduates were surveyed by 
telephone or mailed questionnaire. Although 96 percent of the 
respondents were generally satisfied with their educational 
experience at UKCM, five specific areas were identified where 
graduate responses implied a need for improvement in the educational 
process: (1) patient risk assessment and wellness education; (2) 
student evaluation; (3) integration of basic and clinical content; 
(4) issues relating to the cost of medical care; and (5) the use of 
compute' technology. Contains four graphs and two tables. (MDM) 
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Abstract 

To collect baseline information from the last graduating class of the 
University of Kentucky to have undergone the traditional curriculum (prior to 
the implementation of the RWJ grant), we composed a survey of nine general 
questions about graduates' medical education experiences. We were able to 
interview 77 of the 94 graduates by telephone and an additional 14 graduates 
returned their surveys by mail (97% overall). Most notably, 96% of the 
responding graduates indicated that they were generally satisfied with their 
UKCM educational experiences. The average percentage of positive responses 
from graduates across all items was 72% and for specific items ranged from a 
low of 31% (Neither the amount nor the quality of instructions need to be 
changed) to a high of 95% (Satisfied with training in communication skills). 
The implications of this study include a need to tighten teaching schedules, 
specifying learning expectations and evaluation procedures more precisely, and 
establishing more appropriate levels of patient-care responsibility with 
"more" clinical experience so that fourth-year students are placed clearly at 
a higher level of sophisticated problem solving than third-year clerks. 
Responses from graduates indicated that the curricular areas on which the RWO 
grant focuses could be improved. To track the impact of curricular changes 
related to the RWJ grant on our students' educational experiences, we plan to 
administer a very similar survey to each of the next four graduating classes. 
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UKCM Graduating Seniors' Satisfication with their 
Medical Education- -May 1994 



Introducti on/Background 

For the past several years, on the national scene, there have been calls 
for substantial reform in medical education. Locally the need for educational 
change has become increasingly apparent. The faculty and students of the. 
University of Kentucky College of Medicine (UKCM) sought an educational 
experience that would address the need for balance between: the acquisition of 
medical knowledge and the development of clinical skills, attitudes, and 
values; the amount of passive learning and the need for intellectual 
interaction; teaching about the diagnosis and treatment of disease and 
teaching about disease prevention and health promotion; and care for the 
individual and concern for the community. 

With the help of a grant from the Robert Wood Johnson Foundation (RWJ), 
UKCM has responded to perceived educational needs by planning, funding, and 
implementing a new medical curriculum. The new curriculum increases the 
amount of active student learning, reduces dependency on passive methods for 
the transfer of information, promotes self-assessment and life-long learninq, 
increases the variety of teaching and learninq approaches, and integrates the 
teaching of medical science and clinical practice throughout the curriculum. 

There have been so many changes in the educational program that a program 
evaluation approach has been designed to track the changes as well as their 
immediate effects and long term impact. An important part of these evaluation 
procedures is the collection of baseline information from graduates in the 
class of 1994 who were unaffected by the educational reformation. 

Purpose 

Our purpose for conducting the senior exit survey was to obtain opinions 
from graduates regarding their satisfaction with their medical education. We 
selected a broad range of items to sample elements of the curriculum that 
would be experienced by our medical students in both the traditional and the 
RWJ years. We hope to show that graduates' opinions will become more positive 
across time on those experiences targeted by the objectives of the RWJ grant 
(e.g., medical care cost reduction, computer usage, testing, feedback, etc.). 
We also hope that we see more favorable ratings of overall satisfaction of our 
graduates' educational experiences at UKCM. 

Method 

We constructed a nine-item survey (see Table 1 for the wording of the 
items) to be administered by telephone. The design of the survey required 
interviewers to prompt each graduate to indicate whether or not the question 
was true for them. Graduates' comments were recorded by the interviewer. We 
randomly assigned each graduate to one of four groups, so that we could 
compare groups to determine bias should non-response rate be a problem. Prior 
to making our telephone interview attempts, we sent each graduate a letter 
explaining that we would be calling them within two to three weeks to ask 
about their educational experiences. We made up to five calls to each 

?raduate to obtain a telephone interview. Following the fifth attempt at a 
elephone interview, we sent a letter and a copy of the survey to each non- 
responding graduate asking them to return the survey by mail. 
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Results/Discussion 

In what seomed like an eternity of waiting for the "last" completed 
survey to be returned, with one completed survey arriving in the mail each day 
for eight days (and the last arrived five weeks laterl), we stopped entering 
data into our data base when we had a 97% return rate. We computed chi square 
tests for response bias among the four groups of graduates and among graduates 
interviewed by telephone or returning a mail survey and found no significant 
effects due to interviewer or type of survey method. In addition, we computed 
a one-way analysis of variance on the average percentage of positive responses 
to items obtained from each graduate by interviewer group and by type of 
return (telephone versus mail) and found no significant differences for these 
comparisons. 

We present in three graphs the percent of graduates responding positively 
to each of the items for all graduates (Graph 1), for graduates by interviewer 
group (Graph 2), and for graduates by survey type (Graph 3). We used the 
percentages of graduates making a positive response to each item to arrange 
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smallest percentage of graduates responding positively to the 
include as a part of the display the average percentage of pos 
for all items. The legend for all three graphs (Table 1) prov 
of each item and the percentages for each interviewer group, al 
percentages for all graduates for each item, arranged in the order that the 
items appear (left to right) on each graph. We also present (in Graph 4) our 
estimates of the lowest and highest percentages possible for each item based 
on a 91% (86 of 94 graduates) return rate. 

Of interest is our comparison of pairs of survey items to each other. We 
used McNemar's test of correlated proportions which indicates if changes of 
percentages of graduates responding positively to one item and negatively to 
another item, and vice-versa are significant. These results are displayed in 
Table 2. 

This table may be used to answer the question "Are there significant 
differences between graduate changes in responses for any given pair of 
items?" Item pairs denoted by a "00" indicate significant difference at or 
beyond the .001 level (for example, 96% in Item 1 with 79% in Item 3). One 
item pair (Item 1 with Item 5) was significant at the .01 level. Item pairs 
where there was little spread between two percentages (e.g.. Item 1 and Item 
7, 96% and 95% respectively) were not significantly different from each other 
(ND). 

Even though a large percentage of graduates (96%, Item 1) were generally 
satisfied with their educational experiences at UKCM, graduates' responses, as 
presented in Table 2, imply that specific experiences could have been better. 
For example, the significant differences indicated in between Item 1 (Overall 
Satisfaction) and Item 3 (Student Evaluation System) would suggest that 
graduates feel the evaluation system should be improved. In all, there were 
five specific areas where graduate responses imply a need for improvement in 
the educational process: patient risk assessment & wellness education; student 
evaluation; integration of basic science and clinical content; issues relating 
to the cost(s) of medical care; and, use of computer technology. These five 
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areas are currently receiving major emphasis in the revised Kentucky medical 
curriculum, and the results of this study indicate support for curricular 
revisions that address these issues. 

6raduate Comments 

The summary of interview responses from members of the Class of 1994 is 
based on the written record as prepared by the interviewers as well as their 
recollection of additional data presented by the interviewees. We have 
arranged the questions and comments in the same order as the results displayed 
in the graphs along with the percentage of graduates responding positively. 

Item 1 (96% Satisfied). Overall, were you qenerally satisfied or 

generally dissatisfied with the educational 
experiences you had at UKCH? 

Although generally satisfied with their educational experience at UKCM, 
1994 graduating seniors did suggest the need for improved integration of 
curricular topics and particularly overall organization of the academic 
endeavor. There was some feeling that administration needed to be tightened 
so that teaching schedules particularly in the clinical years, were met and 
course administrative procedures followed. Graduates noted that the fourth 
year should provide for additional patient care responsibility up one notch 
from what was expected as a third year clerk. The opportunity for primary 
care experience was well received. Graduates were positive about their 
treatment as students although there was some feeling that certain learning 
experiences fostered excessive competition. Overall, the faculty was viewed 
as having done a good job. 

Item 7 (95% Yes). Are you satisfied with the training you have received 

regarding development of effective skills for 
communicating with patients and members of a health 
care team (i.e., other doctors, nurses, PAs, etc.)? 

Most graduates were pleased with their training in the development of 
effective skills in communication. There was some level of comment about the 
need to extend these skills to include not only patients but other members of 
the health care team as well. The issue here has to do with the roles of non- 
MD health care providers and how clinical students should or should not 
respond. A few graduates noted that they could have used some help in 
improving their written patient care presentations. Overall, however, the 

?raduates noted that this was one area that received substantial attention and 
hat interaction with patients was eased considerably by the time devoted to 
developing interviewing skills during the first two years of instruction. 

Item 2 (91% Yes). Overall, was the medical knowledge, clinical skills 

and professional attitudes you were expected to learn 
made clear? 

Graduates emphasized the need to clarify student responsibilities in the 
clinical years. Frequently, graduates stated or implied that thev had to 
find out what was expected of them on their own. This was accomplished by 
adapting to whatever a particular clinical service required. There was some 
feeling that a more rigorous orientation to each clinical service would have 
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proved useful as well as more attention to the teaching of particular clinical 
skills. Again, there was the recommendation that additional clinical examples 
be utilized in basic science teaching so that the relationship of basic 
science material to the clinical setting could be mere easily recognized. 
Defining the course and clerkship expectations more completely would probably 
prove useful . 

Item 5 (81% Yes). Do you feel you are well prepared to deal with issues 

related to patient risk assessment and patient 
wellness education? 

The response to this question emphasized that the relevant issues were 
stressed in family practice and other primary care preparation and to some 
extent in preventive medicine but not in other clinical areas. Graduates 
felt they needed to know much more than they did about risk assessment. 
Wellness and nutrition were also emphasized as needing a defined focus of 
instruction. The overall view was that risk management and wellness were 
handled in a sporadic fashion and needed to be integrated in the curriculum so 
that the material and its implications would become a recognized part of each 
students' clinical thinking and problem solving. Graduates clearly believe in 
the importance of these areas to the medical curriculum but are concerned that 
they are not being sufficiently and consistently covered. 

Item 3 (79% Yes). Based on the student evaluation system used at UKCM, 

are you satisfied overall with the self-knowledge you 
presently have regarding your strengths & weaknesses? 

Graduates viewed the evaluation system utilized at UKCM as only modestly 
helpful. "Numbers" provided on the evaluation forms were felt to be 
subjective and not always reflective of real problems. Graduates felt that 
evaluation procedures needed to identify specific areas of weakness and that 
this, in any event, should be done at least raid-way through the course or 
clerkship. Most graduates suggested that they were their own best evaluators 
deciding, as they moved through the educational program, what they did or 
didn't know. There was some focus on whether or not attendings who provided 
evaluations in the clinical years really knew the people they were assessing 
or were dependent on other attendings and/or residents for this information. 
In summary, most graduates paid modest attention to the evaluation numbers 
(with the exception of surgery) but did read the written comments with care. 

Item 4 (66% Yes). Do you feel that the basic sciences & clinical 

sciences were presented in appropriate support of each 
other during your medical education, so that you were 
enabled to solve biomedical problems typical of your 
clinical care experiences? 

Comments generated by this question suggest that the bridge between the 
basic sciences and clinical experiences requires repair. Graduates had much 
to say about the not always clear relationship between basic science material 
and its application in later years. Many respondees felt that synthesis was 
left up to the students and that perspectives of the basic science instructors 
differed significantly from that of the clinicians. Thus, what was stressed 
early on might not be of consequence later in the educational process. 
Graduates cited the need to identify and elaborate on the specific use of 
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basic science information in the clinical context. They felt that more often 
than not they were left to their own devices in making the connection. 
Physical diagnosis and pharmacology were seen as courses critical to 
demonstrating the connection of the first two years of the medical curriculum 
to the second two years. 

Item 6 (59% Yes). Do you feel you are well prepared to deal with issues 

related to reducing medical cs/e costs? 

Graduates feel unprepared in this area. Although costs are mentioned 
from time to time during the clinical years, it is also apparent that many 
tests and procedures are administered where the results may be of doubtful 
use. On the other hand, graduates said that clinical faculty did try to 
discuss medical costs, especially in surgery. Graduates felt the need for 
some preparation in medical economics and management, noting that they are 
entering a business world about which they know very little. As is true with 
responses to other questions, students pick-up information sporadically. Some 

?raduates feel they know a little about cost-containment whereas others feel 
hey know nothing at all. Once again, a consistent presentation of basic 
issues in cost-containment and medical economics seems called for. Graduates 
say that they know little about medical economics and that this area should be 
included in the curriculum, particularly relevant in these days of changes in 
the health care system. 

Item 8 (51% Yes). Has your medical education prepared you to use 

computers and/or other information management 
technologies in your future education and medical 
practice? 

This was a marginal area for most 1994 graduates. Most said they didn't 
learn much about computer usage while at UKCM. What they did learn came from 
family practice and surgery. Some said they learned computer skills in the 
computer laboratory while others noted that what they knew they had learned 
before coming to medical school. There was ambivalence about literature 
searches as a vehicle for learning about computer usage in the practice of 
medicine. Because of the variegated educational background of our students, 
there is a wide sweep of experience or lack thereof with computers. A 
consistent approach to mastering computer technology as a learning tool in 
managing medical school curriculum content is probably a necessary educational 
goal for the majority of UKCM students. 

Item 9 (31% No). Do you feel there are subjects about which you should 
have received more or better instruction during your 
time here as a medical student? 

In general, graduating students responding to this question seemed to 
want a somewhat more "practical" clinical experience with a "bridge" between 
didactic learning experiences and the actual care of patients. Graduates are 
concerned about improving, if not establishing, the basic science-clinical 
bridge. They note that seeing relationships requires an ongoing rather than 
sporadic teaching effort. 

There were many and varied suggestions for content improvement in the 
curriculum. Many graduates interviewed saw themselves as being deficient in 
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starting IVs, doing deep lines, reading chest X-rays, having experience with 
EKGs, interpreting blood specimens, running ventilators and drips. The^e 
concerns merged into the issue of whether they had sufficient basic experience 
in patient management and follow up. Some complained that there was not 
sufficient time devoted to learning about common diseases or indeed time to be 
on the wards. Along with a concern about clinical procedures knowledge, 

graduates also noted that there was little time devoted to learning about 
usiness aspects of medicine, patient care costs and insurance issues. 

With respect to specific subject matter areas, graduates wanted 
additional teaching in dermatology, emergency medicine, nutrition, adolescent 
medicine and radiology. A number of graduates noted that pharmacology should 
include more instruction in drug side effects and that neurology teaching 
should be strengthened. 

Conclusions 

Although we have focused on areas of the curriculum that graduates felt 
we could improve, it is important to note that the average percentage of 
positive responses from graduates across all items was 72% and that 96% of the 
graduates indicated that they were generally satisfied with their UKCM 
educational experiences. However, the implications of this study for making 
improvements may be summarized in a number of arenas, most of which deal with 
the clinical years. For example, there appears to be a need to tighten 
schedules. Instructors who are called away from teaching sessions need to 
ensure that back-up faculty are available to cover their <issignments. Also, 
there is a need to establish levels of patient-care responsibility so that 
fourth-year students are placed clearly at a more sophisticated problem 
solving level than third-year clerks. Clinical clerks need to be monitored 
continually to establish the highest possible skills in written and oral 
communications concerning patient care. Almost all of the graduates wanted 
"more" clinical experience. Providing students practical experience with 
patients and procedures is required and should be an ongoing part of clinical 
preparation. Frequently, this means organized activities to learn clinical 
procedures for which graduates will be responsible as residents. In short, 
graduates want their medical education clearly and cleanly organized and 
administered. Although they understand that everything cannot be covered, 
they also feel' that the learning goals should be crystal clear and that the 
instructional methodology should be effectively linked to those goals. 
Clerkships also need to be as precise as possible with respect to learning 
expectations and evaluation procedures. We do not seem to have hit upon an 
evaluation system with which our students feel comfortable. They complain 
that providing "numbers" does not necessarily r.ake the system objective and 
rather than hearing about strengths and deficiencies after a lapse of time, 
more timely one-on-one interaction with instructors is desirable. 

Finally, as mentioned above, there are five areas where graduate 
responses suggest that improvement in the educational process is needed: 
patient risk assessment and wellness education; student evaluation; 
Integration of basic science and clinical content; issues relating to the 
costs of medical care and; use of computer technology. These areas are 
currently receiving major emphasis in the revised Kentucky medical curriculum, 
and the results of this study indicate graduate agreement with the curricular 
revisions that have been made. 
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Survey of Graduatina Senior8--May 1994 
University of Kentucky College of Hedicine 



Percent of Graduates Responding Positively 



Interviewer Groups 



AM 



DM 



DW 



RJ 



ALL 
STUO- 

ITEM ENTS Item Content 



100 100 95 90 1 96 Overall, were you generally satisfied or generally dissatisfied with the 

educational experiences you had at UKCM? 

95 95 95 94 7 95 Are you satisfied with the training you have received regarding development of 

effective skills for coMnunicating with patients and MMbers of a health care team 
(i.e., other doctors, nurses, PAs, etc.)? 

95 85 90 94 2 91 Overall, was the medical knowledge, clinical skills and professional attitudes you 

were expected to learn Made clear? 

89 75 81 81 5 81 Do you feel you are well prepared to deal with issues related to patient risk 

assessment and patient wellness education? 

74 '80 81 81 3 79 Based on the student evaluation system used at UKCM, are you satisfied overall 

with the self-knowledge you presently have regarding your strengths t weaknesses? 

75 71 70 72 ALL 72 All Items Combined 

58 75 57 71 4 66 Do you feel that the basic sciences ( clinical sciences were presented in 

appropriate support of each other during your medical education, so that you were 
enabled to solve biomedical problems typical of your clinical care experiences? 

63 50 52 68 6 59 Do you feel you are well prepared to deal with issues related to reducing medical 

care costs? 

53 50 57 45 8 51 Has your medical education prepared you to use computers and/or other information 

management technologies in your future education and medical practice? 

53 25 24 26 *9 31 (X No) Do yoo feel there are subjects about which you should have received more or 

better instruction during your time here as a medical student? 
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91 Nurtser of Students Responding 
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INTERVIEWER GROUPS 

AM « Amy Murphy- Spencer 
DM ' David Musick 

DW ' Amy Murphy- Spencer & David Musick 
RJ = Roy Jarecky 
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Significance of Differences in Proportions of UKCM Graduates 
Respondinq Positively-Negatively to Pairs of Survey Items 
Using NcNemar's Correlated Proportions Test 
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* ND « No Difference between proportions of graduates responding 

positively-negatively to pairs of items 
** Leading decimal point not printed 



0 



ERIC 



Graph 1 




Graph 2 



O) 
03 



I . 
I I 

CD 

C/30 

C c 

03 "c 
03 

CD"" 

«+- 

o 

13 




o 



o 
o 



c 



ERIC 



mail ox XisAUisoj §uipuods3^ luaojaj 



Graph 3 




V - Graph 4 




